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Literacy Interventions (OELI) in Indiana's elementary schools. This study 
examined the impact of funding on urban elementary schools, using surveys of 
funded and comparison schools in 1997-98, 1998-99, and 1999-2000. When the 
effects of types of funding were examined in regression analyses, the 
findings were: Reading Recovery funding and OELI funding were associated with 
lower retention rates in urban schools in Indiana; Connected-Text Approaches 
to literacy instruction, which are integral to Reading Recovery and other 
interventions, were also associated with lower retention rates; having 
Certified Specialists (i.e., trained Reading Recovery teachers) was 
associated with lower special education referral in urban schools; and book 
distribution to parents and paired reading between parents and children were 
associated with lower special education referral in urban schools. When the 
effects of the types of interventions that schools implemented were examined 
in a regression analysis, the findings were: Literacy Collaborative (a 
program that combines Reading Recovery with classwide reform) was associated 
with lower retention rates (grades 1-3) in urban schools; and Certified 
Specialists (i.e., Reading Recovery) were associated with lower special 
education referral rates and retention rates in urban schools. These findings 
indicate that the ELIGP has had a significant, measurable impact on 
educational outcomes in urban schools in Indiana. They indicate that ELIGP 
has contributed to the effectiveness and efficiency of urban education in the 
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Abstract 



The Early Literacy Intervention Grant Program (ELIGP) supports training in 
Reading Recovery and provides grants for Other Early Literacy Interventions (OELI) in 
Indiana’s elementary schools. This study examines the impact of funding on urban 
elementary schools, using surveys of funded and comparison schools in 1997-98, 1998- 
99, and 1999-2000. When the effects of types of funding were examined in regression 
analyses, the findings were: 

• Reading Recovery funding and OELI funding were associated with lower 
retention rates in urban schools in Indiana. 

• Connected-Text Approaches to literacy instruction, which are integral to Reading 
Recovery and other interventions, were also associated with lower retention rates. 

• Having Certified Specialists (i.e., trained Reading Recovery teachers) was 
associated with lower special education referral in urban schools. 

• Book distribution to parents and paired reading between parents and children 
were associated with lower special education referral in urban schools. 

When the effects of the types of interventions that schools implemented were examined 
in a regression analysis, the findings were: 

• Literacy Collaborative (a program that combines Reading Recovery with class- 
wide reform) was associated with lower retention rates (grades 1-3) in urban 
schools. 

• Certified Specialists (i.e. Reading Recovery) were associated with lower special 
education referral rates and retention rates in urban schools. 

These findings indicate that the ELIGP has had a significant, measurable impact on 
educational outcomes in urban schools in Indiana. They indicate that ELIGP has 
contributed to the effectiveness and efficiency of urban education in the state. 



Research-Based Reading Reform: The Impact of State-Funded Interventions on 
Educational Outcomes in Urban Elementary Schools 



With passage of the Reading Excellence Act , many states now have the 
opportunity to develop programs that support local efforts to improve early reading. It 
has long been known that young children who attend urban schools are at greater risk of 
not learning to read compared to students attending other types of schools (Slavin, 1991; 
Snow, Bums, & Griffith, 1998). However, there is little agreement about what types of 
reading interventions work best in urban schools. Much of the research receiving 
attention from policymakers has emphasized explicit approaches to instruction (e.g., 
Foorman, Fletcher, Francis, Mehta, & Schatschneider, 1998) and has been criticized 
because it promotes political agendas (Allington & Woodside-Jiron, 1999) and uses 
biased research methods (Taylor, Anderson, Au, & Taffy, 2000). While the researchers 
whose research, funded by the National Institute of Child and Human Development, is at 
the center of the storm have defended their methods and arguments (Foorman, Fletcher, 
Francis, & Schatschneider, 2000; Mathes & Torgesen, 2000), the whole controversy 
creates ambiguity and uncertainty for reformers in states and urban districts. Specifically, 
the following questions merit consideration in states and urban school districts: 

• Should urban schools follow the path toward adopting explicit approaches that 
emphasize phonological awareness, an approach that is widely advocated by 
the conservative politicians who are advocates for fundamental reform? 

• Should urban schools adopt research-based interventions with comprehensive 
designs and an established research base (e.g., Success for All or Reading 
Recovery)? 





• Should states provide funds to urban schools, as a means of encouraging them 
to research the options that they think will work best in their setting? 

Such questions must be considered when states design interventions aimed at 
improving early reading opportunities for urban school children. In this chapter, we use a 
database on early reading programs in urban schools in Indiana to examine the impact of 
state-funded interventions on rates of special education referrals and retention in early 
primary grades. These outcomes are closely linked to the success of schools in teaching 
early reading skills. First, we review the claims about early reading interventions made 
by various education reformers in early reading. Then we describe the research method 
we used in this analysis and present the results. We conclude by examining the claims of 
various reformers. 

A Critical Review 

While the idea of research-based reform sounds compelling, it can be difficult in 
an area like early reading to discern the types of reforms that merit consideration by local 
schools. A first step in untangling this knot involves examining the specific claims made 
by various types of reforms. Three types of claims are used in the research and policy 
literatures: 

• Claims made by researchers about methods or combinations of methods that 
can improve early reading opportunity. 

• Claims made by advocates of specific reforms that argue for a particular 
combination of program features, organized in cohesive programs, that are 
intended to improve early reading opportunity. 



• Claims by policymakers for specific program approaches (e.g., requiring a 
particular method [e.g. phonics] for all schools versus categorical grant 
funding). 

Researchers ’ Claims 

The debates about early reading instruction are situated along a continuum with 
two poles: one that emphasizes meaning and context and another that emphasizes letter- 
sound relationships, or phonics (Chall, 1967). In the current context, much of the 
controversy centers around two arguments. 

Perhaps the most widely cited research on phonological awareness and the 
alphabetic principle is by Barbara Foorman and her colleagues (Foorman, Fletcher, 
Francis, & Schatschneider, 1998). Recently, in response to critics, Foorman and her 
colleagues summarized their argument: 

However, we do maintain that there are some instruction principles that teachers 
and schools can use to enhance the reading achievement of at-risk children, and 
that it makes sense to demonstrate effective implementation of these instructional 
principles before investing in more complex solutions. (Foorman, Fletcher, 
Francis, & Schatschneider, 2000, p. 27) 

Thus, while Foorman and her colleagues do not explicitly oppose comprehensive 
approaches to early reading improvement, they do argue for a specific method as a first 
priority. They base this argument on research that found: 

Controlling for differences in age, ethnicity, and verbal IQ, we found that children 
in the direct code (DC) approach improved in word reading at a faster rate and 
had higher word recognition skills in April than children receiving the implicit 
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code (IC) approach (either research-based IC or district’s standard IC). More 
importantly, children in all instructional groups with higher phonological 
processing scores in the beginning of the year demonstrated improvement across 
the year. (p. 29) 

Thus, this argument rests on research that shows students who have instruction in direct 
coding learn to read words faster and that students who have skills in phonological 
processes learned to read faster. Based on these findings, Foorman and her colleagues 
conclude that schools should first emphasize direct instructional approaches. 

There is a large array of counter arguments on early reading. One researcher who 
has recently received wide attention, Barbara Taylor and her colleagues (Taylor, 
Anderson, Au, & Taffy, 2000) argue that the Foorman research overlooked the context of 
education in the schools in which Foorman’s research was situated. Specifically, they 
argue: 

Literacy research documents an array of practices important for struggling 
beginning readers. ... These practices influence systematic instruction in word 
recognition, carefully selected texts, repeated reading, guided writing, regular 
assessment of pupil progress, extra time in reading, one-on-one tutoring, strong 
home connections, and ongoing staff development, (p. 24) 

This argument has two specific components. One component is that the specific 
practices that are emphasized are closely aligned with a holistic approach emphasizing a 
literature-rich approach to early reading instruction. A second component of their 
argument is that comprehensive intervention is needed rather than an approach that 
emphasizes a single method first, such as direct instruction. 



There is a clear contrast between these two positions. One argument is that direct 
instruction is of primary importance and should be implemented before other approaches 
are tried (e.g., Foorman, Fletcher, Francis, & Schatschneider, 1998, 2000). The counter 
argument is that comprehensive approaches to intervention are appropriate (Taylor, 
Anderson, Au, & Taffy, 2000). Thus the claims of the two camps have clear and direct 
policy implications. 

The Claims of Reform Advocates 

Much of the research on early reading interventions has been conducted by 
advocates of specific reforms or by evaluators of these reforms. This complicates the 
situation facing urban educators who are interested in choosing a research-based 
intervention that works for their school. It is important that school leaders understand the 
features included in specific reforms and the confirmatory research base underlying each 
one. The implicit claim of most reading reforms is that implementation of the reform can 
improve early reading, as well as reduce the number of students who are retained or 
referred for special education services. Therefore, it is important that schools consider 
the features of reforms, to see if they complement the instructional methods they use. 

Five of the reforms that have well-defined, comprehensive approaches to early reading 
intervention are outlined below, along with descriptions of the types of services available 
to schools in Indiana. 

Reading Recovery is a comprehensive approach to early reading that emphasizes 
one-on-one tutoring. In Indiana, Purdue University provides training for teachers 
interested in Reading Recovery. The state-funded training for teachers interested in 
Reading Recovery through the Early Literacy Intervention Grant Program (ELIGP) and 
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Title I paid the direct operational costs in some of the schools that chose this option. It 
emphasizes: 

• Ongoing professional development for reading specialists; uses a Vygotskian 
developmental philosophy; 

• Involvement of parents in reading with their children; 

• Using meaning in context to teach phonemes (rather than direct decoding), 
along with aligned drills and creative method; and 

• A sequence of literature books to promote the development of reading skills 
by students who are at risk of not learning to read (Bardzell, 1999b). 

A confirmatory research base indicates that children who complete the program make 
normal process in their education (Askew & Frasier, 1994; Lyons, 1994, Pinnell, Lyons, 
DeFord, Bryk, & Seltzer, 1994; Pinnell, DeFord, & Lyons, 1988). However, one set of 
researchers found that including more direct instruction within Reading Recovery 
improves the speed at which students complete the program (Iverson & Tunmer, 1 993), a 
finding that has created controversy for the proponents of this intervention method. 

First Steps is a comprehensive classroom-wide intervention that was chosen by 
some schools applying for school-wide grants through Indiana’s ELIGP. An intervention 
method developed originally in Australia, First Steps: 

• Has options for ongoing support and networking available in the United 
States; 

• Emphasizes a developmental approach and student empowerment; 

• Provides training for parents in literacy instruction; 
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• Uses holistic and literature-rich approaches, but includes phonics in some 
versions of the intervention; 

• Uses systematic diagnostic procedures and formative evaluation, along with a 
literature-rich environment (Manoil, 1999). 

Thus First Steps is an approach that is situated in the literature-rich tradition but that 
includes a number of systematic features that involve teachers in assessing the progress 
children make in learning to read. While indicating that First Steps has an impact on 
students, teachers, and schools (e.g., Australian Council for Education Research, 1993a, 
1993b; Deschamp, 1995), the research is largely descriptive. 

Four Blocks is a classroom- wide intervention developed by Cunningham (1991) 
that strikes a balance between holistic and direct methods. This method includes: 

• A philosophical approach that emphasizes both phonological awareness and 
student empowerment; 

• Both literature-rich and direct approaches to early literacy instruction; and 

• Basal readers and a literature-rich environment. (Manoil & Bardzell, 1999) 

In the form originally proposed by Cunningham, Four Blocks did not include 
professional development or parental involvement. This program does have modest 
confirmatory research (Cunningham, Hall, & Defee, 1991, 1998). The Indiana 
Department of Education has offered workshops on Four Blocks. In addition, some 
schools have received grants through ELIGP to implement the program. 

Literacy Collaborative is a new classroom-wide reading intervention designed to 
complement Reading Recovery. In Indiana, Purdue University provides ongoing 
professional development in support of the Literacy Collaborative and a few schools have 



received funding for this intervention through ELIGP. The program includes the 
following features: 

• University training for both pullout (through Reading Recovery) and ongoing 
professional development in support of Literacy Collaborative; 

• A philosophy that is developmental and emphasizes student empowerment 
and creating learning communities; 

• A comprehensive approach to instruction that includes diverse writing, oral, 
and decoding methods; and 

• A comprehensive, literature-based and systematic curriculum for all students 
in grades 1-3 (Bardzell, 1999a). 

Literacy Collaborative has only recently been developed by Ohio State University (1998) 
and lacks any confirmatory research base. However, initial studies of implementation in 
Indiana indicate that the method promotes improvement in student outcomes (St. John, 
Manset, Chung, Simmons, & Musoba, 2000). 

Success for All is a comprehensive school -wide intervention approach (Slavin, 
1991) that has a well-developed early literacy component. This well-documented method 
includes the following features: 

• Systemic ongoing professional development and support; 

• A developmental approach that emphasizes phonological awareness and 
whole language; 

• An emphasis on building parent awareness and skills development; 

• A systematic approach to instruction that emphasizes cooperative learning as a 
means of teaching decoding in both explicit and meaning-oriented ways; and 
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• A balanced approach that emphasizes classroom-wide activities, small groups, 
and pullout. (Bardzell, 1999c) 

The research on the effects of Success for All is extensive. Most of the research uses 
specially selected comparison schools and word recognition tests (e.g., Madden, Slavin, 
Karweit, Donlan, & Wasik, 1991; Madden, Slavin, Karweit, Livermon, & Donlan, 1989), 
although there is also some confirmatory research by independent researchers (Ross & 
Smith, 1994). 

The outline of reforms above illustrates the array of comprehensive approaches to 
early reading reform available to urban schools, if they have funding. Some emphasize 
pullout, others classroom-wide interventions, and still others combine approaches. The 
reforms take different approaches to instruction in decoding, the topic that underlies the 
debate among researchers outlined above. Reading Recovery, Literacy Collaborative, 
and First Steps emphasize a meaning-oriented approach to teaching decoding, an 
approach Foorman and colleagues (Foorman, Fletcher, Francis, & Schatschneider, 2000) 
strongly criticized. In contrast. Success for All and Four Blocks take a balanced 
approach, emphasizing context-free decoding, the approach Foorman and colleagues 
(Foorman, Fletcher, Francis, & Schatschneider, 2000) favor. 

Political Claims 

Two policy paths are evident in the latest round of policy conversations that 
consider the research base on reading. One approach, adopted by Texas, Washington, 
California, New York, and Wisconsin, has been to require the teaching of phonics 
(Allington & Woodson- Jiron, 1999; Taylor, Anderson, Au, & Taffy, 2000). This 
approach to policy takes the claim of Foorman, Fletcher, Francis, Mehta, & 
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Schatschneider (1998) quite literally. That is, these researchers essentially claim that 
direct phonics instruction is a necessary first step and legislators in these states have 
taken action, requiring this instructional approach. The implicit claim taken in this 
approach is that requiring all schools to use an explicit approach to phonics instruction 
will increase the number of students who learn to read and make normal educational 
progress. 

An alternative approach is also now available to states to provide funding to 
schools for comprehensive reading interventions. This approach is being used in 
Kentucky, for example, as part of the state-funded program using federal funds from the 
Reading Excellence Act. In Indiana, the Early Literacy Intervention Grant Program 
(ELIGP) has used annual categorical grants to fund reading interventions in schools since 
1997. One component of the program funds teachers seeking training for Reading 
Recovery. The other component includes a wide variety of interventions categorized as 
Other Early Literacy Interventions (OELI). The implicit claim of categorical grant 
programs is that comprehensive interventions can promote educational progress. 

Research Approach 

This paper presents an empirical study of the impact of ELIGP in urban schools, 
adapting a method developed from a statewide study of the intervention program (St. 
John, Manset, Hu, Simmons, & Michael, 2000). Our purpose in conducting this study is 
to untangle evidence related to the three types of claims outlined above. 

We used three years of survey data on schools in Indiana, collected as part of a 
comprehensive evaluation of the Early Literacy Intervention Grant Program in Indiana. 
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Below we describe the survey we used, the logical model used in the study, and the 
instruction and related factors developed from the longitudinal study. 

Survey and Response 

The Early Literacy Intervention Survey 2 included questions about the types of 
reading programs that were implemented, amount of time per day spent on reading, 
features of the early reading program, number of students referred and retained, and 
enrollment data that could be used to impute special education referral and retention 
rates. In addition, we had access to a state-level database with information on test scores. 

The survey assessed the frequency of use of nine organizational and structural 
features (ability grouping, basal readers, child-initiated learning centers, independent 
reading, one-on-one tutorial, pullout instruction, small groups, systematic evaluation, and 
trade books). It also assessed the frequency of use of ten classroom instructional methods 
(Big Books, cooperative learning, creative writing and/or essays, drama, emergent 
spelling, paired reading, phonics, reading aloud, reading drills, and 
worksheets/workbooks). For these frequency-of-use questions, survey participants were 
asked to respond using a five-point scale from 1 for “never” to 5 for “everyday” for both 
the current year and the prior year by grade level (K, 1 , 2, 3). The survey also asked 
whether five types of professional development processes (certified training, certified 
specialist, in-service workshops, networking, and opportunity for collaboration) and five 
features related to parent involvement (book distribution, family literacy, paired reading, 
parent conference, and parent volunteers) were used in kindergarten through grade three. 




2 Copies of the Survey can be obtained on request from the Indiana Education Policy Center. A simplified 
version of the instrument that can be used to survey teachers is available on-line (St. John, Manset, & 
Michael, 1999) 



This study reports analyses for three years of surveys of funded and comparison 
schools. The overall response rate across the three years was 61%. Comparison schools 



were half as likely to be surveyed as funded schools. Therefore, comparison schools 
were weighted by 2 to adjust for the probability of being surveyed. The study uses a sub- 
population of schools located in urban school districts. 

Statistical Methods 

The study used descriptive statistics, factor analysis, and multiple regression. 
Descriptive statistics were used to describe the population characteristics. 

A factor analysis (principal components analysis with varimax rotation) was 
performed using SPSS (version 10.0) for 19 variables related to instructional and 
classroom program features for the entire population. Specifically, the average Likert 
score for each of the three grades (1-3) was imputed for the 19 program features on the 
survey related to instruction and structure/organization. Missing items were replaced with 
mean values. A factor-loading minimum of .40 was used for inclusion of a variable in 
interpretation of a factor, and only factors with eigenvalues greater than 1 .0 were 
interpreted. 

Ordinary least squares (OLS) regression was used to examine the influence of 
predictor variables on two outcomes. We present R 2 , plus three levels of significance 
(.01 , .05, and .10) for each predictor variable. Since .1 is only a moderately significant 
association, we make note of this moderate association in the text, so the reader will not 
place undue emphasis on this statistical relationship. 
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Model Specifications 

We have two versions of the multiple regression models, one assessing type of 
funding and the other assessing type of intervention. In addition, we consider two 
distinct outcomes with each mode: rate of special education referral and rate of retention 
in grade level. Initially, we used sequential regressions, adding blocks of related 
variables for each outcome. The blocks of variables included: 

• School Characteristics: The average ISTEP+ score, the percentage of students 
receiving free or reduced lunch, the percentage of minority students, and 
school locale: urban or rural. 

• Funding Type or Intervention Type: In the analyses of funding types, we 
considered Reading Recovery (RR), OELI-1-3 (an OELI intervention in 
grades 1 through 3), OELI-K (OELI in kindergarten), OELI-FDK (an OELI 
intervention for full-day kindergarten), and OELI-preK (an OELI intervention 
for pre-kindergarten). In the analysis of intervention types, we considered 
Reading Recovery 3 , Success for All, Literacy Collaborative, Full-Day 
Kindergarten, First Steps, Even Start, Accelerated Schools 4 , and Four Blocks 
Method. The two Type variables (funding or intervention) are not mutually 
exclusive. 

• Professional Development: This variable designates whether each of the 
following is included in professional development: requiring reading teachers 
to be certified, bringing in certified specialists for training sessions, in-service 

3 This variable coding included schools with Reading Recovery whether or not they were funded through 

ELIGP. 
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workshops, opportunities for teachers to network with teachers in other 
schools, and opportunities for teachers to collaborate within the school on 
reading instruction. 

• Instruction and Related Factors: For this factor, we included the scores on 
each of the nine structural/organizational features and ten classroom 
instructional methods. 

Instructional and Related Factors 

In order to reduce the large number of program features related to instruction and 
the organization of reading programs at the grade level, we performed a principal 
components factor analysis of the instructional and structural/organizational features. 
(The factor analysis is presented in St. John, Manset, Chung, Simmons, & Musoba, 
2000 .) 

The Connected-Text Approaches factor includes independent reading, cooperative 
learning, creative writing, emergent spelling, paired reading (student-to-student), and 
reading aloud. Schools that make use of these methods combine techniques that engage 
students in the learning process. 

The Explicit/Direct Approaches factor combines basal readers, phonics 
instruction, reading drills, and worksheets/workbooks. Schools that emphasize explicit 
approaches utilize systematic approaches to teaching the components of language and 
reading. 

The Child-Centered/Expressive Approaches factor combines child-initiated 
learning centers. Big Books, cooperative learning, and drama. These instructional 

4 The ELIGP did not fund any Accelerated Schools, although this intervention type was discussed in 
documents disseminated through the program (St. John & Bardzell, 1999) and there were a few Accelerated 
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approaches place an emphasis on the development of the whole child and peer 
engagement among children. 

The Ability Group/Pullout Approaches factor combines ability grouping, one-to- 
one tutoring, pullout instruction, and small groups. Schools that use these techniques 
place more emphasis on classifying children and accelerating the learning of some, while 
addressing developmental needs of others. 

The Trade Books Approaches factor combines trade books and Big Books, but de- 
emphasizes basal readers. Schools that use this approach emphasize using texts that are 
literature-based and engaging for students, rather than emphasizing the structured 
elements of reading programs organized around increasing levels of difficulty. 

Of these factors, the variables included in the Explicit/Direct Approaches factor 
are closely aligned with the direct approaches advocated by Foorman and colleagues 
(Foorman et al., 1998, 2000). The other factors would seem more closely aligned with 
the more comprehensive approaches advocated by Taylor and other reform advocates. 
Limitations 

This study has a few limitations that merit consideration by readers. First, our 
analyses consider school-related outcomes rather than individual outcomes. While most 
reading research focuses on individual students, we felt it was important for the funding 
agency to understand whether their funding influenced school-related outcomes. This 
approach, while unusual, is consistent with the ways school outcomes are frequently 
reported and appears appropriate for a policy study of this type. 

Second, the survey queried respondents about program features of each grade 
level, rather than asking teachers to answer questions about their classrooms. We 
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considered this approach appropriate for an initial test of the study methodology. In the 
future we plan to extend the method to include a survey of teachers, which would mean 
we could examine both school-level and classroom-level outcomes. 

Third, we assumed that all schools in the funded and comparison groups had an 
equal probability of returning a survey. This assumption was necessary because of the 
statistical methods used here. This assumption is typical when researchers use survey 
responses in regression models. 

Fourth; for consistency across studies, we used factor scores generated from an 
analysis of the entire population of schools in all locales. Since one of our aims is to 
compare this urban-schools analysis to a similar analysis of the entire population (i.e., St. 
John, Manset, Chung, Simmons, & Musoba, 2000), we decided it was most appropriate 
to use a consistent set of factors. However, the reader is cautioned that new factor scores 
were not calculated for this population. 



Findings 

Sample Characteristics 

The characteristics for the 150 urban schools in the sample are presented in Table 
1 . The average special education referral rate for the schools surveyed was .05, with a 
standard deviation of .03. The average retention rate was .03 with a standard deviation of 
.03. In the average school in the sample, 44% of the students qualified for free or 
reduced lunch and 31% were minority. 
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